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Energy literacy is multidimensional, comprising broad content knowledge as well as 
affect and behavior. Our previous study has defined four core dimensions for the 
assessment framework, including energy concepts, reasoning on energy issues, low- 
carbon lifestyle, and civic responsibility for a sustainable society. The present study 
compiled a series of contextualized question items into a computer-based test (CBT) 
platform to examine students’ energy literacy. Each test unit included a scenario, 
presented via multimedia materials (e.g., text, image, short video, and animation), that 
pertained to real-life situations. Various types of questions were employed (e.g., multiple 
true-false, multiple choice, and short-answer) that required students to construct 
responses and make judgments. A total of 1,711 secondary school students participated 
in this survey. The results indicate that the energy literacy level of Taiwanese secondary 
students is discouragingly low and inter-correlations between the dimensions of energy 
literacy reveal that energy knowledge and behavior are more closely correlated than 
affect and behavior. In addition, the scores on the attitudinal items were slightly higher 
for junior than senior students and the students in the southern region scored higher on 
energy literacy than those in other regions. These findings provide evidence for the 
future development of energy-related educational curricula and materials that can 
improve students' energy literacy and engagement in energy-related decisions. 
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INTRODUCTION 

Energy usage has a profound impact on our standard of living and every major 
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sector of the economy. The recent issue of climate change mitigation demands 
attention on energy efficiency and reducing energy use in order to pursue 
sustainable economic growth (Chen, Huang, & Liu, 2013; Whitmarsh, Seyfang, & 
O’Neill, 2011], In addition, policy proposals relying on individuals' voluntary carbon 
reduction highlight the need for at least some level of public understanding of 
energy conservation. In this regard, education plays an important role in 
establishing modes of communication that can improve individuals' energy literacy 
so that energy can be utilized both rationally and efficiently (Dias, Mattos, & Jose, 
2004; Liarakou, Garvrilakis, & Flouri, 2009; Zografakis, Menegaki, & Tsagarakis, 
2008], In general, the objectives of energy education are to develop people's 
awareness about energy crises, to make them understand the energy-environment 
nexus, and subsequently to ensure environmental sustainability of every nation 
(Kandpal & Garg, 1999], The concepts of carbon reduction should also be 
incorporated into curricula and teaching activities in order to improve student 
awareness about energy conservation and related practices (Directorate-General for 
Energy and Transport, 2006], For example, since 2008, Taiwan's government has 
promoted its Sustainable Energy Policy in which the action plans include supporting 
environmental education for energy saving and carbon reduction as well as 
designing and implementing relevant teaching materials (Yeh & Chuang, 2009], 

In recent years, environmental education has risen steadily with the goal of 
fostering environmentally literate citizens (Teksoz, Sahin, & Tekkaya-Oztekin, 2012; 
Spiropoulou, Antonakaki, Kontaxaki, & Bouras, 2007], Since energy literacy has been 
an important agenda for environmental education, energy education shares the 
same rationale with environmental education. In the 1990s, numerous studies were 
conducted to develop frameworks for defining the components of environmental 
literacy (Hungerford & Volk, 1990; Marcinkowski, 1990; Roth, 1992], which 
subsequently guided several national assessments of environmental literacy 
(McBeth, Hungerford, Marcinkowski, Volk, & Meyers, 2008; Negev, Sagy, Garb, 
Salzberg, & Tal, 2008; Shin et al., 2005], These frameworks measured the domains of 
knowledge, affect, cognitive skills, and behavior, which are commonly regarded as 
critical to environmental literacy (McBeth & Volk, 2009], By referring to these 
aforementioned frameworks developed in Western countries, the present article 
compiles its own framework for designing the assessment of energy literacy in 
Taiwan (Chen et al., 2013], Furthermore, it examines the design of a multi¬ 
dimensional assessment and analyzes the test results of a nationwide sample. 

Defining Energy Literacy 

Energy literacy is regarded as an educational effort that helps pave the way 
toward a more energy-secure future by empowering individuals to choose 
appropriate energy-related behaviors throughout their daily lives (DeWaters & 
Powers, 2011], The notion of energy literacy comprises broad content knowledge of 
affective and behavioral dimensions as well as the competency that people need to 
make wise choices and commit to energy conservation (DeWaters & Powers, 2011, 
2012; DeWaters, Qaqish, Graham, & Powers, 2012; DoE, 2012; Lay, Khoo, Treagust, 
& Chandrasegaran, 2013], 

In their survey of New York state, DeWaters and Powers (2011, 2012] developed 
an Energy Literacy Questionnaire to assess secondary students’ energy literacy in 
three core dimensions: cognitive (knowledge], affective (attitudes, values], and 
behavior. Their findings also emphasized that energy literacy embodies not only 
content knowledge, but citizen engagement. Additional studies (Bodzin, Fu, Peffer, & 
Kulo, 2013; Lay et al., 2013] also utilized an Energy Literacy Questionnaire to 
investigate the levels of energy literacy among 8 th grade students in Malaysia. 
Furthermore, it was shown that energy literacy not only strives for behavioral 
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change, but also for enabling individuals to make thoughtful decisions based on 
science (DoE, 2012], These previous findings emphasized the importance of 
practical energy-related knowledge, decision-making skills, value judgments, and 
ethical and moral dimensions related to energy conservation. In order to develop a 
framework for energy education in Taiwan, the Chen and colleagues (2013] adopted 
the ideas of energy literacy and carbon capability. We also applied the Analytical 
Hierarchy Process [AHP] method with a panel of experts to determine the core 
dimensions (Table 1], which basically conform to the common constructs of 
environmental literacy (i.e., knowledge, attitude, behavior, and civic engagement]. 
However, the present study placed greater emphasis on practical knowledge and 
high-level cognitive skills. 


Table 1. Dimensions of environmental literacy and energy literacy (Chen et al., 2013] 


Dimensions 

Environmental Literacy 

Energy Literacy 

Knowledge 

Knowledge 

Knowledge 

Understandings 

Skills 

Affective 

Interest 

Sensitivity attitude 


Sensitivity 

Locus of control 

Self-efficacy 

Behavior 

Responsibility 

Intentions 

Involvement 

Action 


Energy Education in Taiwan 

Due to the lack of local energy resources, effective energy use has become one of 
the most important issues in Taiwan (Chuang & Ma, 2013; Huang & Wu, 2009], 
Moreover, the Taiwanese government has continually revised its energy policy in 
favor of sustainable energy use (Huang & Wu, 2009; Tsai, 2005], In November 1979, 
the Energy Commission was established by the Ministry of Economic Affairs (MOEA] 
to develop and implement a national energy policy (MOEA, 2012], In 2007, an action 
plan titled, "Energy Saving and Carbon Reduction," in accordance with the 
Sustainable Energy Policy Framework, was approved and promoted by the 
Executive Cabinet (Yeh & Chuang, 2009], The goal of this plan was to improve 
energy efficiency and conservation, and subsequently reduce carbon dioxide (CO 2 ) 
emissions. However, individual behavior change and public engagement are still key 
factors in promoting and implementing an energy-related policy. 

Since 1979, energy education has been promoted in various countries such as the 
United Kingdom, the United States, and Australia (Hsu, Huang, Fu, & Teng, 2010], In 
line with the international trend, the Ministry of Education in Taiwan has funded 17 
centers and established an energy education program targeted at elementary and 
lower secondary schools (Tsai, 2005], In addition, the National Science Council 
implemented a national-level project to invest human capital in the research and 
development of an effective energy education program. The success of an energy 
education program requires a comprehensive assessment to ensure that its 
educational objectives align with the criteria for energy literacy, which include not 
only cognitive aspects, but also affective and behavioral characteristics. 
Furthermore, since energy education programs must be tailored to local, regional, 
and international priorities and requirements, it is necessary to establish clear 
educational criteria for these programs. Therefore, our previous research aimed at 
developing a framework of energy literacy that targeted the ability to make 
informed judgments regarding energy use as well as take effective actions related to 
energy management (Chen et al., 2013], Our study also found that the dimensions of 
"civic responsibility for a sustainable society" and "low-carbon lifestyle" were 
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considered as the most important energy education goals. Due to the local, regional, 
and international priorities and requirements for energy education, this study 
designed an assessment that can help improve energy education by understanding 
the energy literacy of secondary students. 

Contextualized Assessment for Energy Literacy 

In order to ensure that students attain the desired proficiency levels, a suitable 
assessment should be developed to evaluate the effectiveness of energy education 
programs. In real-life situations, students must use different cognitive processes to 
understand various components of an event, especially when the event is related to 
complex environmental problems. However, conventional instruments were limited 
to paper-and-pencil tests with single, closed items that measured students' 
knowledge and attitudes. According to Lee (2004], using a contextualized test makes 
it possible to measure examinees' understanding of the material from various 
perspectives. This format has been used to assess complex cognitive processes by 
the Organization for Economic Co-operation and Development [OECD] and by the 
International Association for the Evaluation of Educational Achievement [IEA] 
(Mullis et al., 2007; OECD, 2002], Routitsky and Turner [2003] also endorsed the 
importance of using various test item formats in order to accommodate the full 
range of student abilities typically sampled in the Program for International Student 
Assessment [PISA], 

Substantial studies have been performed on energy literacy surveys, but there is 
still a critical lack of comprehensive tests for energy literacy. In addition, the 
availability of a suitable instrument that measures broad energy literacy has been 
limited or non-existent (DeWaters & Powers, 2011] since such literacy indicates 
having the required knowledge and skills as well as the ability to make informed 
decisions when participating in society (Harlen, 2001], We argue that the findings of 
these survey assessments do not represent energy literacy levels since they lack a 
context in which the respondents can solve energy-related problems in a holistic 
manner. Moreover, based on a non-curricular approach, the test format is coherent 
with the general theoretical framework of this study (OECD, 2003, 2006], Therefore, 
literacy-related assessment is conducted as a series of contextualized questions to 
evaluate, from various perspectives, the complex process of respondents' 
understanding of the material (Monseur, Baye, Lafontaine, & Quittre, 2011], 

Information and communication technology (ICT] has become a crucial 
component in the education process (Alabi, Issa, & Oyekunle, 2012], The computer- 
based test (CBT] has been considered as a special type of ICT that has become 
increasingly important in various fields of competence assessment (Wirth, 2008], 
Complex problem solving is an essential skill for students when dealing with 
complex and dynamic systems (Frensch & Funke, 1995] and their performances can 
be measured by a CBT, even in large-scale assessments (Wirth & Klieme, 2003], A 
CBT can also be used to assess tacit knowledge regarding procedures/strategies that 
cannot be easily verbalized and difficult to assess through conventional pencil-paper 
tests (Buchner, Funke, & Berry, 1995], The framework of energy literacy, as 
described earlier, includes the following components that students are expected to 
possess: the ability to acquire energy-related knowledge; the ability to make 
informed judgments about energy saving; and the ability to take actions of energy 
management in complex situations. In order to assess students’ energy literacy in a 
more comprehensive manner, the present study designed an assessment tool that 
requires students to answer a series of question items after comprehending the 
scenarios presented via multimedia materials (e.g., text, image, short video, and 
animation]. For this assessment design, the CBT provided both accessibility and 
flexibility. 


204 


© 2015 iSER, InternationalJ. Sci. Env. Ed., 10(2], 201-218 




Assessing multidimensional energy literacy 


For a shift toward more energy conservation in society, it is necessary to 
understand the level of energy literacy among students as an indicator of the 
effectiveness of energy literacy efforts (Fah, Hoon, Munting, & Chong, 2012], In 
order to measure energy literacy among secondary students in Taiwan, we 
developed an instrument based on the test format adopted by the PISA. This 
instrument included multiple types of questions that pertained to real-life situations 
and required the students to construct responses and make judgments. The purpose 
of this study was to answer the following research questions: 

1. How do Taiwanese students perform in regard to the four dimensions of 
energy literacy (i.e., energy concepts, reasoning on energy issues, low-carbon 
lifestyle, and civic responsibility for a sustainable society]? 

2. Is there any correlation among individual variables, the four dimensions, and 
overall energy literacy? 

3. Is there any relationship among individual variables and the students' 
performance on the energy literacy assessment? 

The answers to these questions should provide evidence for the future 
development of energy-related educational curricula and materials that can improve 
students' energy literacy and engagement in energy-related decisions. 

MATERIALS AND METHODS 

Survey Instrument 

Contextualized assessment may be viewed as the most appropriate approach for 
evaluating complex processes such as science comprehension (Monseur et al., 
2011], which requires students to employ different cognitive processes to 
understand various components of the same text for a non-curricular approach. The 
OECD has applied this format for its PISA tests since 2000 (Monseur et al., 2011], 
and the IEA used it for its first reading comprehension test in 1971 (Elley, 1994; 
Mullis et al., 2007; Mullis et al., 2003], In consideration of the non-curricular 
approach and the various aspects of energy literacy, the present study compiled a 
series of contextualized question items into a CBT platform in order to explore 
energy literacy among Taiwanese secondary students. In the following sections, we 
provide a description of the energy literacy framework used for development and 
assessment, illustrate the test format and the process of item development, and 
present some samples of the items. 

Assessment framework for energy literacy. Table 2 summarizes the dimensions 
and indicators defined in this study. The four core dimensions are described as 
follows: 

1. Energy concepts: In order to actively participate in decision making for a low- 
carbon society, an energy-literate individual should possess energy concepts, 
which include a systematic understanding about sources, uses, and development 
of energy, and its costs and benefits, 

2. Reasoning on energy issues: An energy-literate person should have the ability to 
evaluate the reliability of information sources that help ascertain how to use 
energy effectively, and make judgments on energy-related issues. 

3. Low-carbon lifestyle: Individual behavior is important for government policy on 
carbon reduction and energy-related issues. For a sustainable society, 
individuals must choose to adapt their daily lifestyles to contribute to carbon 
reduction and energy conservation. 

4. Civic responsibility for a sustainable society: Individuals' awareness about 
climate change leads to their engagement in energy conservation activities and a 
shift towards a sustainable society. In the energy literacy assessment, the 
essential qualities of energy literacy include the ability to make informed 
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judgments about the use of energy and take effective actions with regard to 
energy management. 


Table 2. Dimensions and indicators of the energy literacy framework (Chen et al., 2013] 
Dimension Indicator One should be able to: 


1. Energy concepts 


2. Reasoning of 
energy issues 


3. Low-carbon 
lifestyle 


4. Civic 

responsibility for 
a sustainable 
society 


1-1 Awareness of sources, uses, 
and development of energy 

1-2 Possessing a systematic 
understanding about energy 

1- 3 Understanding energy costs 
and benefits 

2- 1 Exploring international and 
local energy issues 

2- 2 Judging and evaluating 
information about energy-related 
issues 

3- 1 Identifying carbonless 

technology and action plans 

3- 2 Carrying out sustainable 
(green] consumption 

4- 1 Awareness and self-efficacy 


4-2 Civic engagement 


Acknowledge issues related to energy sources and 
resources 

Interpret the discovery, development, and use of energy 
resources 

Recognize that the different sources of energy and the 
ways of using them have different costs, implications, risks, 
and benefits 

Assimilate the interpreted current events in an 
international or local context relevant to energy issues 

Analyze and assess objective and reliable information 
relevant to energy issues 

Adapt appropriate lifestyles that contribute to solving 
global energy problems 

Consider the impact of energy consumption patterns 

Be aware of the impact of personal energy-related 
choices on the environment; believe that personal actions 
can lead to a change 

Be acquainted with energy-saving and carbon-reducing 
activities and engaged in decision making on energy-related 
issues 


Instrument format. The instrument included a student information survey and 
an energy literacy assessment. The survey consisted of items related to the students' 
backgrounds, such as gender, grade, school location, numbers of learning activities 
attended, and perceptions of energy conservation and carbon reduction. The items 
regarding perceptions were based on a five-point Likert scale (with one neutral 
response] ranging from 1 (strongly disagree] to 4 (strongly agree]. The energy 
literacy assessment contained a series of contextualized test units with each unit 
dealing with a real-life problem or issue. Each test units began with a stimulus text 
taken from authentic sources such as newspapers, blogs, and books about energy 
issues. The students were then asked to answer a series of items related to the 
problem or issue. The items were arranged in groups of independently scored items 
based on a common stimulus, which enabled us to utilize contexts that realistically 
reflected the complexity of life situations as well as efficiently used the time allotted. 
Four types of item formats were used in the assessment and each item aligned with 
one of the indicators in the energy literacy framework. Approximately one-third of 
the items were selected-response (multiple-choice] items that required the selection 
of a single response out of four options, while another third included multiple true- 
false items and short-answer questions. The multiple true-false items required the 
respondents to answer multiple "yes/no" or "true/false" questions, while the short- 
answer questions required a written response that included short explanations 
and/or justifications. For the attitudinal items, another third of the questions were 
based on a four-point Likert-type. In this study, the terms "cognitive items” and 
"attitudinal items" are employed to distinguish between these two types of items 
where necessary. 

Figure 1 presents a contextualized test unit titled, "Carbon Footprint,” which 
includes a graph and diagram as well as various types of test questions. A 
commentary is also provided to explain how several items that test different 
processes and concepts can be presented in one unit. The stimulus material for this 
unit contained various factors that contribute to carbon footprints. The 
contextualized test unit required the respondents to use the given information to 
support a particular course of action and to recognize the carbon emission of the 
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fuel and water purchased as well as their daily carbon footprint coverage. Two 
attitudinal items in this test unit assessed the respondents' disposition regarding 
how their energy consumption patterns impacted the environment and examined 
their attitudes toward engagement with carbon reduction in their daily lives. Figure 
2 displays a contextualized test unit titled, "Cherish the Environmental Resources," 
which includes an animation as well as various types of test questions. The stimulus 
material for this unit used the animation to describe students' consumption 
behaviors for their breakfast and recycling behaviors. The contextualized test unit 
required the examinees to use the given information to recognize the choices of 
product packaging. Two attitudinal items in this test unit assessed students’ 
disposition to saving energy with families and friends in their everyday lives. 


• 4* - smusemti&zrmtm s ns • wm* 
1 - y&mm - • i&ms* _ (s tub) • gras* 


tetBsnz&is’raitatiiiai * 

WEI 99.11.18/11:4 





BffSUXSBSHBNBBSm (m ■ 

s/mra- 


Carbon Footprint 

Mr. Wang noticed a statement on his fuel invoice that read, "The CO 2 
emission of this fuel purchase is approximately 32 kg." This 
statement made him aware of the high level of CO 2 that his fuel was 
emitting! Additionally, his home electricity and water bills provided 
similar calculations of the carbon footprint, such as "The CO 2 
emission calculated from your electricity usage in this statement is 
approximately 72 kg. Please save electricity to help reduce CO 2 
emissions and the impact of global warming" and "The CO 2 emission 
calculated from your water usage in this statement is about 2 kg. 

Please conserve water.” (as described on the invoice below). With 

such information on CO 2 emissions, users are reminded of the carbon 
emissions a ssociated with their daily lives. 

*# ( 

Water Coosumptkm Charge Payment Notice & Receipt >' 

WBl 99.11.18/11:4* 

847 jVm ^ 1 ~ 

Referring to the carbon footprint coverage chart below, which of the 
following statements regarding carbon footprints is incorrect ? 

A. Carbon footprints can help consumers understand the effect and 

impact of their consumption choices on the environment. 

B. It is necessary that the calculation of the carbon footprint 

includes direct and indirect CO 2 emission throughout the life 
cycle of the commodity. 

C. If a commodity has no carbon footprint labeling, then it does not 

produce CO 2 emissions. 

D. The carbon footprint is a referral benchmark for measuring 

carbon emissions. 



Response format: 
Simple multiple-choice 


• .'Mr-*!' Response format: 

n ■ iMHtsHWMttM- Order 


Based on the product life cycle, please calculate the carbon footprint 
of the following actions and arrange them in descending order, from 
the action with the highest carbon footprint to the action with the 
lowest (e.g., A > B > C). 

A. Dining in a restaurant and using self-prepared tableware. 

B. Preparing your own meal at home and eating more vegetables 

and less meat. 

C. Bringing fast-food meals home and using the plastic bags and 

disposable utensils provided by the store. 



SMKtWKA!WiKAn«ESI< £tHt it “if f. 

Attitudinal items: 

4-point Likert scale 


**W it Tit 





To what extent do you agree with the following statements? 

A) I will remind my family members to compare the carbon 
footprints of the products that they use. 

□Strongly Agree nAgree oDisagree nStrongly Disagree 

B) I will take part in the "Power Saving Guru" contest. 
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□Strongly Agree nAgree ^Disagree nStrongly Disagree 

Figure 1. A contextualized test unit—Carbon Footprint 





? again* • 


Response format: 
Constructed response 
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Response format: 
Multiple true-false 


nat* ms.. ? 

SHflUHS-SSSSf • 


mm Rt THS IWfflS 


Cherish the Environmental Resources 

mm 


Animation: Yu and 
Maruco talk about 
their breakfast and 
the waste generated 



Which of Maruco's behaviors are improper in 
terms of cherishing environmental resources? 
Please list three. 

Which statements about 
product and packaging 
materials (paper or plastic) 
TRUE FALSE are true? 

The product recycling 
o o process does not consume 

energy. 

The recycling rate for 
plastics can be up to 100%. 
Recyclable resources, such 
o o as paper and plastic, can be 

called "renewable energy." 
Referring to the animation, to what extent do you 
agree with the following statements? 


WtAUfflS ■ OBMnRMamSSffM 


JiSBS m TH* SFSTBt 


Attitudinal item: 4-point 
Likert scale 


A) I will persuade my friends to use resources 
more efficiently. 

□Strongly Agree nAgree ^Disagree nStrongly 
Disagree 

B) When shopping with my family, I will discuss 
whether we need to purchase the products. 
□Strongly Agree nAgree ^Disagree nStrongly 
Disagree 

Figure 2. A contextualized test unit- Cherish the Environmental Resources 


Item development. In the process of designing the test items, we involved 
experienced secondary school teachers to ensure that the context of the items were 
as close as possible to the students' real-life experiences and levels of energy 
literacy. A series of teacher workshops was held to accomplish the following steps: 
1] introducing the assessment framework of energy literacy; 2] presenting the 
advantages of contextualized question items; 3] brainstorming possible scenarios 
for designing questions; and 4) developing contextualized test units through 
teamwork. 

A total of 40 teachers from various subjects/areas were recruited to participate 
in the workshops after which they were asked to design at least one test unit. As a 
result, 20 units were submitted for consideration. The contextualized test units were 
separately presented for review to a panel of experts specializing in environmental 
education, science education or energy technology. Various criteria, such as the 
overall quality of the unit, the amount of revisions required, and indicators in 
framework coverage, were used to determine which units to retain. The six selected 
units (49 question items) were then modified into a paper-and-pencil test that was 
administered to 200 junior high school students. Following Wainer's (1990) 
suggestion, our pilot study first employed the classical test theory to analyze 
Cronbach's alpha values, and item difficulty and discrimination, and then adopted 
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the item response theory (IRT] to identify and compare the fitness index (including 
deviance value, BIC, and A 2 ] between the one-dimensional model and the multi¬ 
dimensional model in order to justify the validity and reliability of the question 
items. Each subscale appears to be internally consistent, as indicated by Cronbach’s 
alpha values, which ranged from 0.70 to 0.82. The item difficulty values (from 0.21 
to 0.84] and item discrimination values (from 0.24 to 0.53] indicate that the 
question items were appropriately designed for assessing students' knowledge and 
cognitive skill. Based on the item response theory (IRT], the results show that the 
fitness index of the one-dimensional model (deviance value = 7737.80; BIC = 
7933.84; A 2 = 2216.08] is slightly lower than that of the multi-dimensional model 
(deviance value = 7739.79; BIC = 7972.92; A 2 = 2578.03], Multi-dimensionality not 
only improves the reliability of individual tests, but it also accurately estimates the 
correlation between dimensions since measurement error has been factored into 
the estimation process (Wang, 2004], The results from the pilot study supports that 
this multi-dimensional assessment is a valid and reliable quantitative measure for 
assessing high school students' energy literacy. 

Sample 

A total of 1,711 secondary school students were drawn from 18 junior high 
schools and 13 senior high schools in the central, north, south, and east regions of 
Taiwan. These schools were informed of the research objectives and the 
requirements of technical support after which they agreed to participate in this 
study. One or two classes of students (depending on the size of the school] were 
randomly selected as the sample and the selected students were gathered in the 
computer classroom at their school in order to complete the online test. The final 
data was gathered from 974 junior and 737 senior high school students. As shown in 
Table 3, 51.3% of the participants were male while 48.7% were female, and the 
majority were in the 10 th grade (23.7%] with the remaining students distributed 
among the 8 th grade (22.1%], the 7 th grade (22.0%], and the 11 th grade (18.2%]. 
Finally, the students came from various regions with a broad range of 
socioeconomic backgrounds (north, 41.1%; central, 26.3%; south, 25.1%; and east, 
7.5%]. 


Table 3. Characteristics of participants 


Characteristic 


% 

Characteristic 

% 

Regional 

North 

34.07 

Grade 

7 th 

25.69 


Central 

30.01 


8th 

19.63 


South 

29.72 


9 th 

11.20 


East 

6.20 


10* 

23.56 

Gender 

Female 

50.82 


nth 

15.22 


Male 

49.18 


12* 

4.70 


Reliability and Validity of the Instrument 

To ensure the reliability and validity of the scale items, we used Cronbach’s alpha 
to evaluate the internal consistency, which exceeded the recommended minimum of 
0.7 (Nunnally, 1978], Confirmatory factor analysis, using structural equation 
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modeling (SEM] software (LISREL 8.51], was performed to test the validity and 
reliability of the assessment. The factor loadings were statistically significant with 
values of 0.82 to 0.22. In addition, the composite reliability was 0.69 and the average 
extracted variance (AVE] was 0.40. These values, combined with acceptable 
Cronbach's alpha coefficients of 0.85, supported the reliability of the assessment 
(Hair, Anderson, Tatham, & Black, 1999], Finally, the discriminant validity among 
the dimensions of energy literacy was tested using Fornell and Larcker's [1981] 
criteria in which the square root of the AVE should be greater than the correlations 
between the construct for satisfactory discriminant validity. Thus, it was determined 
that all dimensions exhibited satisfactory discriminant validity. 

As previously noted, instrument validity was supported through various methods 
such as drawing items from existing energy and environmental research, 
administering the instrument to a panel of experts in energy, environment, and 
science education. Moreover, the values from Cronbach’s alpha as well as the results 
of SEM and the AVE have satisfied the criteria of reliability and validity. 
Consequently, it was determined that the instrument was reliable and valid for 
assessing secondary students' energy literacy 

Data Analysis 

The students' questionnaire responses were converted into numerical scores. 
The cognitive items were assigned one point for each correct answer and zero 
points for each incorrect or blank response. In addition, the attitudinal items were 
converted to numerical values according to a predetermined preferred direction of 
response in order to calculate the summated rating totals. 

In order to examine the relationship between the dimensions of energy literacy 
and overall energy literacy, inter-correlations between the average student scores 
on the items and overall energy literacy were calculated with Pearson's correlation 
coefficients. The effects of students’ gender, grade, and region on the dimensions of 
energy literacy and overall energy literacy were investigated using multiple 
regression analysis. 

RESULTS AND DISCUSSION 

Student Performance on the Four Dimensions of Energy Literacy 

In order to answer the first research question, the performance summaries for 
the four dimensions of energy literacy are presented in Table 4. According to the 
passing grade for the tests (commonly set at 70%], the results in this study indicate 
that this sample of Taiwanese secondary students have a positive attitude towards 
energy conservation and carbon reduction. However, their performances were 
discouragingly low with regard to the dimensions of "low carbon lifestyle" and 
"reasoning on energy issues.” It was also evidenced that two items had the lowest 
percentage of correct responses in the assessment: "judging and evaluating 
information about energy-related issues" and "reasoning on energy issues." For 
instance, the item of Test Unit 2 titled, "Fossil Fuel Energy," required students to use 
the given information to choose a correct description of the goal and situation of the 
2012 COP16 in Cancun, Mexico (25% correct responses]. The item of contextualized 
Test Unit 3 titled, "Geothermal Energy," required students to choose the correct 
information about the characteristics of geothermal energy (32% correct 
responses]. The results of the test show that the students failed to identify the 
information about current events relevant to energy issues and that they lacked the 
ability to analyze and evaluate the energy-related issue information. Furthermore, 
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the findings show that the students also required more understanding about energy 
resources. 

The two items related to the indicator, "awareness of sources, uses, and 
development of energy," under the dimension of "Energy Concepts," also had fewer 
correct responses. For instance, the item of contextualized Test Unit 4 titled, "New 
Era—Biomass Energy," asked students to choose the correct answer for biodiesel 
(38% correct responses]. Moreover, the item of contextualized Test Unit 3 titled, 
"Geothermal Energy" asked students to determine the characteristics of air 
conditioning using geothermal energy (42% correct responses]. It is noteworthy 
that the students lacked a systematic understanding about alternative energy and 
the ability to obtain new information about such energy. They also had limited 
ability to gather and analyze information about local and international energy- 
related issues. Evidently, a lack of knowledge is a common theme in participants' 
understanding of energy conservation, which is similar to the findings of DeWaters 
and Powers (2011], 

Table 4. Summaries for student performance on four dimensions of energy 
literacy by demographic backgrounds_ 


Dimension 



l.Energy 

concepts 

2 .Reasoning on 
energy issues 

3.Low-carbon 

lifestyle 

4.Civic 

responsibility 

Overall 

energy 

literacy 

Gender 

Male 

71.17 

68.44 

66.15 

82.21 

72.00 

Female 

70.91 

68.56 

66.23 

82.13 

71.96 

Region 

North 

70.52 

68.11 

65.69 

82.05 

71.59 

Central 

71.17 

68.44 

66.15 

82.21 

72.00 

South 

70.78 

68.22 

65.85 

82.11 

71.74 

East 

70.96 

67.89 

65.15 

81.88 

71.47 

Grade 

7 th 

70.30 

67.89 

65.46 

82.09 

71.44 

8th 

70.70 

68.22 

65.77 

82.11 

71.70 

gth 

70.48 

68.11 

65.54 

82.14 

71.57 

10* 

70.96 

68.44 

66.00 

82.11 

71.88 

nth 

70.61 

68.11 

65.77 

82.00 

71.62 

12th 

70.39 

68.11 

66.15 

81.71 

71.59 


Note: Each value in this table has been converted into a percentage of possible full score. 


Relationships among Measured Variables 

Table 5 presents the correlation coefficients among the dimensions of energy 
literacy and other surveyed variables. The result of the correlation analysis shows 
that the dimensions of energy literacy are positively correlated with one another 
and there is a relatively high relationship between "energy concepts" and "low 
carbon lifestyle," with an obtained r as large as .62. It was also found that there are 
weak relationships between "civic responsibilities for a sustainable society" and the 
other dimensions of energy literacy. Conversely, "energy concepts" is most likely 
associated with the other dimensions of energy literacy, especially "low carbon 
lifestyle," which is considered as a behavioral aspect that emphasizes personal 
choices of appropriate lifestyles and practices of carbon reduction. This finding is in 
accordance with earlier models of environmental behavior, which assumed the 
widely held position that knowledge leads to changes in attitudes and values, which, 
in turn, fosters action or behavior (Fah et al., 2012], 

The results displayed in Table 5 indicate that the students’ perceptions on the 
importance of energy conservation and carbon reduction are positively correlated 
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with all four dimensions of energy literacy. However, according to the values of the 
correlation coefficient, only the relationship between perceptions and attitudes 
("civic responsibilities for a sustainable society"] is significantly recognized (r = .54], 
This finding supports the assumption that students' perceptions of individual and 
collective efforts toward energy conservation may be revealed in their positive 
attitudes about environmental responsibility (Lawrenz & Dantchik, 1985], Another 
surveyed variable, the number of environmental learning activities students had 
experienced, unfortunately, shows no or weak correlations with all of the energy 
literacy dimensions. Based on our observations, most of the learning activities are 
provided by school curricula and they mainly focus on conceptual learning of energy 
technology and related knowledge, which may be insufficient for improving 
students' comprehensive understanding of energy issues. Nevertheless, we argue 
that the issue-based teaching approach can be incorporated into learning activities 
in order to formulate students' abilities to evaluate and assess information in regard 
to energy issues. 


Table 5. Mean, standard deviations, and correlation coefficients of measured 
variables 


Variables 


1 

2 

3 

4 5 6 7 

Energy Literacy 

1.Energy concepts 

- 





2.Reasoning on energy issues 

.49“ 

- 




3.Low-carbon lifestyle 

.62“ 

.43“ 

_ 



4.Civic responsibility for a sustainable 
society 

.16“ 

.14“ 

.18“ 

- 


5.Overall energy literacy 

.83“ 

.74“ 

.84“ 

.40“ - 

Perception 

6.Students' perceptions of energy 

conservation and carbon reduction 

.11“ 

.11“ 

.15“ 

.54“ .26“ - 

Experience 

7.Number of learning activities attended 

.02 

.04 

.07“ 

f— * 

4^ 

O 

GO 

O 

vD 


"Significant at the p < .01 level. 

Regression Analysis for Estimating Predictors To Energy Literacy 

A multiple regression analysis was conducted to test the relationships between 
individual variables and energy literacy. The categorical variables, such as gender, 
grade, area, and region, were dummy variables used to predict the dimensions of 
energy literacy. Since the number of learning activities attended did not correlate 
with energy concepts and reasoning on energy issues, it was excluded from Model A 
and Model B of Table 6. The results show statistically significant relationships 
between the respondents’ characteristics and the dimensions of energy literacy. In 
each regression, the variance inflation factors [VIF] were considerably less than 5.0, 
thus indicating that multi-collinearity was not a problem (O'Brien, 2007], Table 6 
shows the results of the multiple regression analysis that examined the 
relationships between respondents' characteristics and the dimensions of energy 
literacy. Model A indicates that the respondents' characteristics (including gender, 
region, grade, and perception of the need for energy conservation] are significant to 
"energy concepts." Model B shows that the variables of region, grade, and perception 
are the significant predictors for "reasoning on energy issues," while Model C 
indicates that all of the respondents' characteristics (except for gender] are 
significant to "low-carbon lifestyle." Finally, Model D shows that region, number of 
learning activities attended, and perception of the need for energy conservation are 
significant to "civic responsibility for a sustainable society," while Model E is similar 
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to Model C in that the respondents' characteristics (excluding gender] are significant 
to overall energy literacy. 

Table 6. Regression analysis for four dimensions and overall energy literacy 



Model A b 

Model B b 

Model C b 

Model D b 

Model E b 


P 

P 

P 

P 

P 

Gender 






Female 3 






Male 

0.14* 

-0.02 

-0.04 

-0.05 

-0.02 

Region 






North 3 






Central 

0.15* 

0.16* 

0.09* 

0.13* 

0.18* 

South 

0.30* 

0.22* 

0.19* 

0.07* 

0.29* 

East 

-0.02 

0.05 

-0.06* 

0.03 

-0.02 

Grade 






T 






8 

0.11* 

0.08* 

0.10* 

-0.02 

0.11* 

9 

0.18* 

0.09* 

0.06* 

-0.01 

0.13* 

10 

0.42* 

0.27* 

0.38* 

-0.03 

0.41* 

11 

0.15* 

0.13* 

0.13* 

-0.04 

0.15* 

12 

0.15* 

0.10* 

0.15* 

0.01 

0.15* 

Number of learning 



0.09* 

0.09* 

0.10* 

activities attended 






Students’ perceptions of 






energy conservation and 

0.11* 

0.10* 

0.13* 

0.53* 

0.24* 

carbon reduction 






R 1 (Adjust R 1 ) 

0.21(0.21) 

0.11(0.10) 

0.16(0.15) 

0.32(0.31) 

0.25(0.24) 

F 

46.35* 

20.15* 

29.13* 

78.55* 

54.89* 


’Significant at p < .05 level. 
a Reference group 

b Dependent variables: Model A, “energy concepts’’ 
Model B, "reasoning on energy issues” 

Model C, "low-carbon lifestyle" 

Model D, "civic responsibility for a sustainable society" 
Model E, overall energy literacy 


The results show that the respondents' characteristics are positively associated 
with energy literacy. The variable of gender is merely significant to energy concepts 
since the male students outperformed the female students in regard to such 
concepts. Region and students' perceptions are positively associated with "low- 
carbon lifestyle" and overall energy literacy. For example, students in the central 
and southern regions scored higher than those in the northern region. However, in 
regard to "low-carbon lifestyle," students in the northern region scored higher than 
those in the eastern region. Furthermore, students in the 7 th grade performed worse 
on the knowledge and behavioral dimensions than those in grades 8-12, except for 
"civic responsibility for a sustainable society." There are positive relationships 
between the number of learning activities attended and the overall energy literacy 
and "low-carbon lifestyle" and "civic responsibility for a sustainable society." 

A t-test was conducted to further understand the differences between grades and 
the performance of the dimensions and overall energy literacy. Based on the 
structure of the education system in Taiwan, the students are divided into two 
groups in terms of their stage of school education: grades 7-9 (junior high students] 
and grades 8-12 (senior high students]. The result of the t-test also indicates that 
the senior high students (n = 899] show significantly greater performance on the 
knowledge (energy concept: t = -14.38, p < .001; reasoning on energy issues: t = - 
9.339, p < .001] and behavioral dimensions (low-carbon lifestyle: t = -13.18, p < 
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.001] and overall energy literacy (t = -11.09, p < .001] compared to the junior high 
students (n = 1167], This result may be used to interpret the goal of energy 
education, which familiarizes students with energy conservation and carbon 
reduction in order to improve their cognition of energy conservation and related 
practices (Zografakis et al., 2008], In this case, senior high students might possess 
more advanced energy-related knowledge since they have been engaged in energy- 
conservation concepts through each stage of energy education. This finding is in 
accordance with the result that "energy concepts" and "low carbon lifestyle" were 
strongly positively correlated. 

For "civic responsibility for a sustainable society," there was no statistical 
difference between junior and senior high students, despite the fact that the scores 
on the attitudinal items are slightly higher for junior students than those of senior 
high students (t = -1.073, n. s.]. Similar to the findings from earlier studies (e.g., 
DeWaters & Powers, 2011; Whitmarsh, Seyfang, & O'Neill, 2011], there appears to 
be a value-action gap among students' cognitive, affect, and actions. Although senior 
high students were concerned about the energy problems of society, they apparently 
lacked the actions of engaging in energy-conservation behaviors in their daily lives. 
DeWaters and Powers (2011] clarified that as adolescents become young adults, 
they are less willing to change their habits that consume more energy. In addition, 
the students in higher grades have lower levels of environmentally friendly 
behaviors because the current provision systems are not conducive to such 
practices (Whitmarsh et al., 2011], 

Male students had significantly higher energy-related knowledge than the female 
students, yet there was no correlation in their attitude scores, which is in line with 
the study by Lusardi and Mitchell (2007], Like several other studies (e.g., Barrow & 
Morrisey, 1989; Gambro & Switzky, 1999; Lay et al., 2013], gender disparities were 
found in energy and environmentally related knowledge. Other studies also showed 
general trends of gender differentiation in science achievement and increased 
differentiation as students progressed through school (Clewell & Campbell, 2002; 
Haertel, Walberg, Junker, & Pascarella, 1981], The finding in the present study that 
male students outperformed the female students only on "energy-related 
knowledge" may be similar to the gender difference in science learning achievement. 

It is interesting to note that students in the southern region scored higher on 
energy literacy than those in the other regions, including the northern region, which 
mostly contains Taiwan's metropolitan and urban areas. This result differs from 
other studies in that respondents from urban schools performed better on the 
cognitive skills and attitudes toward energy conservation than their rural 
counterparts (Fah et al., 2012; Lay et al., 2013], 

Personal experiences of the surroundings could raise people's demands for 
environmentally friendly practices, and foster their long-term environmental 
concerns (Chawla, 1999], The study by Davidson, Yantis, Norwood, and Montano 
(1985] also found that the prediction of behavior is associated with personal 
experience. For example, natural disasters, such as typhoons and floods, occur 
frequently in Taiwan and they cause serious damage to properties and the quality of 
life, especially for those in southern region. Thus, the impact of natural disasters can 
be used as teaching cases in school since students might possess awareness of 
environmental issues and then develop their energy literacy in their respective 
environments. Compared to the southern region, the infrastructure in the northern 
region of Taiwan is more advanced. As a result, the students from the northern 
region may not directly suffer from environmental disasters, and for this reason, 
they are more likely to have lower levels of energy conservation-related attitudes 
and practices. Raising awareness of the urgency of energy problems and the need for 
energy education programs might foster actual behaviors in energy conservation 
and carbon reduction (DeWaters & Powers, 2011], 
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CONCLUSION AND RECOMMENDATION 

Educational efforts may help transform people's behaviors toward rational use of 
energy and increase their energy literacy (Dias, Mattos, & Balestieri, 2004; 
Zografakis et al., 2008], The government sectors in Taiwan, such as the Ministry of 
Education [MOE] and the Ministry of Science and Technology [MOST], have 
promoted energy education programs targeted at each stage of school education as 
well as developed various educational programs and evaluation methods for energy 
education through national projects. However, the success of an energy education 
program is in need of an adequate and comprehensive assessment (DeWaters & 
Powers, 2011], Therefore, this study comprehensively evaluated the energy literacy 
levels of secondary students in Taiwan in order to understand the effectiveness of 
the energy education policy. The instrument in this study consisted of 
contextualized test units with various test items to accommodate students' 
comprehensive energy literacy, which included cognitive, affective, and behavioral 
aspects. 

Based on the assessment results, the energy literacy level of Taiwanese 
secondary students is unsatisfactory. Students had limited abilities to evaluate 
information about global energy issues and they lacked knowledge regarding new 
energy resources. Students in grades 8-12 outperformed those in the 7 th grade on 
the knowledge and behavioral dimensions of energy literacy. Furthermore, the 
results of the correlational analyses show that the behavioral dimension is more 
closely correlated with energy knowledge than the affective dimension. It is 
suggested that effective educational programs should target students' abilities to 
possess a basic understanding of energy concepts as well as evaluate and assess 
information in order to actively participate in decision making for a low-carbon 
society. The energy curricula should also incorporate active pedagogical techniques, 
such as the issue-based teaching approach, to increase the diversity of activities for 
engaging students and ultimately improve their civic knowledge, skills, and 
participation (Papadimitriou, 2004], 

Energy education programs should be tailored to local and regional concerns and 
priorities in order to be relevant to learners' living experiences and interests. The 
results indicated that students in the southern region scored higher on energy 
literacy than those in the other regions, thus suggesting that local issues, such as 
impact of natural disasters, can be applied as teaching cases to raise awareness of 
the urgency of energy problems and the impact of decisions regarding energy use. 
Moreover, such issues can foster actual behaviors in energy conservation and 
carbon reduction (DeWaters & Powers, 2011], 

The instrument designed in the present study adopted the recommendation by 
Monseur et al. (2011] in which a format of contextualized items might be suitable 
for assessing complex processes such as scientific literacy. We agreed with the idea 
that a literacy-related assessment should be conducted using a series of 
contextualized questions that examine students' understanding of complex 
materials and processes from various perspectives. More specifically, the instrument 
used in this study contained contextualized test units with various test items, and 
empirical analysis demonstrated its suitability to accommodate the full range of 
student abilities as well as broadly encompass the cognitive, affective, and 
behavioral characteristics of energy literacy. Incorporating the CBT platform also 
allowed the contextualized assessment to be more accessible and interesting to the 
students since students of this generation are generally comfortable with using 
computers and most schools are equipped with sufficient ICT environments. Finally, 
it is suggested that the CBT for energy literacy assessment be conducted in a 
nationwide follow-up study containing longitudinal and multidimensional features 
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of energy literacy for a wider population (e.g., students, teachers, and parents] in 
order to improve our understanding of the factors that influence energy 
conservation behaviors and evaluate the effectiveness of energy education as a 
whole. 
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